Nonlinear scaling in video conferencing

ABSTRACT

A method for video teleconferencing is disclosed. The method comprises receiving a first starting image with a first number of rows and receiving a second starting image with a second number of rows. The images are nonlinearly down-scaled to create a first scaled image and second scaled image respectively. Thus the numbers of rows in the scaled images are less than the number of rows in the starting images. Then, the first scaled image is combined with the second scaled image to create a transmission image for transmission.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a bypass continuation of International ApplicationNo. PCT/US2014/018137, filed Feb. 25, 2014, entitled NONLINEAR SCALINGIN VIDEO CONFERENCING, which claims the benefit of U.S. ProvisionalPatent Application Ser. No. 61/768,587, filed Feb. 25, 2013, entitledNONLINEAR SCALING IN VIDEO CONFERENCING, the disclosures of which arehereby incorporated by reference.

BACKGROUND

Various aspects of the present disclosure relate generally toteleconferencing and more particularly to compression of video forteleconferencing applications.

Conventional conferences involve locating participants at a commonlocation (e.g., a conference room). However, in certain circumstances,it is not possible for all participants of a conference to be co-locatedwithin the same conference room. To address this scenario, videoconferencing systems have been developed that take advantage of computernetworks to transmit video and audio data between two or moregeographically separate locations. Basically, the video conferencingsystem enables the participants at one location (i.e., a local endpoint)to see and hear the participants at a different, remote location (i.e.,a remote endpoint).

In its most basic form, a video conferencing system equips eachphysically separate location (endpoint) participating in a videoconference with a camera, microphone, video display, and speaker. Thevideo camera captures video images of the participants at acorresponding local endpoint. The video conferencing system processesthe captured video images (e.g., using compression) and transmits theprocessed video images to a remote endpoint, where the compressed videoimages are decompressed and displayed. Similarly, the microphonecaptures audio from the local endpoint, which is transmitted to theremote endpoint of the video conference. Analogously, the audio can becompressed for transmission and decompressed for audio playback at theremote endpoint.

The video display enables the participants at the local endpoint to viewvideo captured at the remote endpoint. The display may also be utilizedto view other forms of media, such as images, slide presentations, andvideo clips. Likewise, the speaker enables the participants at a localendpoint to hear audio captured by the microphone at the remoteendpoint. The speaker may also be used to play media, such as audiofiles. As such, participants at each location of the video conferencecan see and hear the participants at the other location, as well as seeand hear any other forms of media presented during the conference.

BRIEF SUMMARY

According to aspects of the present disclosure, video teleconferencingis provided, comprising receiving a first starting image with a firstnumber of rows and receiving a second starting image with a secondnumber of rows. The images are nonlinearly down-scaled to create a firstscaled image and second scaled image respectively. Thus the numbers ofrows in the scaled images are less than the number of rows in thestarting images. The first scaled image is combined with the secondscaled image to create a transmission image for transmission.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is an illustration showing a video conferencing system usingnonlinear scaling, according to various aspects of the presentdisclosure;

FIGS. 2A-2I are block diagrams illustrating nonlinear down-scalingstarting from a camera image to a transmission image, according tovarious aspects of the present disclosure, particularly:

FIGS. 2A and 2B are block diagrams illustrating a starting image;

FIGS. 2C and 2D are block diagrams illustrating applying a dual-axiswarp process to the starting images of FIGS. 2A and 2B, respectively;

FIGS. 2E and 2F are block diagrams illustrating dividing up the imagesof FIGS. 2C and 2D, respectively;

FIGS. 2G and 2H are block diagrams illustrating performing down-scaleprocessing of the images of FIGS. 2E and 2F, respectively;

FIG. 2I is a block diagram illustrating an exemplary combining of theimages of FIGS. 2G and 2H, respectively;

FIG. 3 is an illustration showing an exemplary dual-axis warpingfunction performed concurrently with an exemplary nonlinear down-scalingfunction, according to various aspects of the present disclosure;

FIG. 4 is a flow chart illustrating a method of nonlinear scaling,according to various aspects of the present disclosure;

FIG. 5 is a flow chart illustrating a method for video teleconferencingup to transmitting an image, according to various aspects of the presentdisclosure;

FIG. 6 is a flow chart illustrating a method for video teleconferencingfrom receiving the transmitted image from FIG. 5, according to variousaspects of the present disclosure; and

FIG. 7 is a block diagram of a computer system having a computerreadable storage medium for implementing functions according to variousaspects of the present disclosure.

DETAILED DESCRIPTION

A typical conference room that is set up as a video conferencingendpoint includes a display monitor that is positioned towards a firstend of a conference room table. Participants of a video conference siton either side of the table or otherwise sit around the table so as toface the display. Because the gaze of participants is often directedtowards the display, a video camera is typically placed on top of thedisplay so that an image of the faces (or at least the heads) of theparticipants in the conference room can be captured. The video imagescaptured by the video camera are transmitted to another geographicallyremote endpoint location for display on a corresponding video screen. Ifa second video camera is added to a local endpoint (e.g., where a singlevideo camera is insufficient to capture high quality images of all ofthe participants attending the video conference at the local endpoint)then the complexity of the video conferencing system greatly increases,typically resulting in the requirement of twice the number of codecs,connections and bandwidth usage at all endpoints.

However, according to aspects of the present disclosure, two videoimages may be transmitted by the video conferencing system usingbandwidth required for only one image. As such, in video conferencingsystems that use nonlinear scaling of the present invention and have twocameras per endpoint, the bandwidth to send the images from the cameraswill be the same as if only one camera were used.

According to aspects of the present disclosure, nonlinear multi-bandscaling (as will be described in greater detail below in reference tothe figures) is utilized to process an image captured from a videoconferencing camera such that different portions of the image are scaledwith different scaling functions. For example, sections of the imagethat include more detailed information will be scaled less than othersections of the image. Thus, a method is provided that takes a startingimage, breaks that starting image into sections, and scales the sectionsindividually with different scaling functions. Within this document, theterm “starting image” means “an image before that image is nonlinearlymulti-band scaled.” Thus, a “starting image” is an image that will bemulti-band scaled at a later time and can include images that have alsobeen previously processed (e.g., scaled, warped, cropped, or acombination thereof) and raw images directly from cameras.

Referring to the figures, and in particularly FIG. 1, an illustrationdepicting a video conferencing system 1000 with two endpoints 1100, 1200is shown. Within FIG. 1, reference numerals beginning with 11 xx areassociated with the first endpoint 1100 and reference numerals beginningwith 12 xx are associated with the second endpoint 1200.

The first endpoint 1100 includes two cameras 1110, 1120, each capturingan image (i.e., raw images) 1112, 1122, respectively, of one half of aconference table 1130. The images 1112, 1122 from the cameras 1110, 1120are sent to a frontend system 1140 that nonlinearly down-scales theimages to create multi-band scaled images 1114, 1124. The frontendsystem 1140 further combines the multi-band scaled images 1114, 1124into a transmission image 1142 and sends the transmission image 1142 toa codec (i.e., encoder/decoder) 1150 for transmission over a network1050.

Likewise, the second endpoint 1200 includes two cameras 1210, 1220, eachcapturing an image 1212, 1222, respectively, of one half of a conferencetable 1230. The images 1212, 1222 from the cameras 1210, 1220 are sentto a frontend system 1240 that nonlinearly down-scales the images tocreate multi-band scaled images 1214, 1224. The frontend system further1240 combines the multi-band scaled images 1214, 1224 into atransmission image 1242 and sends the transmission image 1242 to a codec1250 for transmission over the network 1050.

The frontend systems 1140, 1240 may also perform a dual-axis imageequalization process in addition to the nonlinear multi-band scalingprocess. The dual-axis image equalization process is explained ingreater detail in International Application No. PCT/US2012/041135entitled “DUAL-AXIS IMAGE EQUALIZATION IN VIDEO CONFERENCING” filed onJun. 6, 2012 on behalf of Herold Williams and Array Telepresence, Inc.,and the entirety of that International Application is herebyincorporated by reference. When performed, the dual-axis equalization(i.e., dual-axis warping) may be performed serially to the nonlinearscaling or the dual-axis equalization may be performed concurrently withthe nonlinear scaling.

The codecs 1150, 1250 encode the transmission images 1142, 1242,respectively, for transmission over the network 1050. Further, thecodecs may also perform other processing such as compressing thetransmission images. Thus, the codec 1150 of the first endpoint 1100encodes the transmission image 1142 to send to the second endpoint 1200and vice versa.

The codec 1150 of the first endpoint 1100 receives the transmissionimage of the second endpoint from the network 1050 and decodes thereceived image 1162 for a backend system 1160. The backend system 1160divides (i.e., separates) the received image 1162 into scaled images1172, 1182 for two displays 1170, 1180 and performs an inverse nonlinearscaling equation (i.e., up-scaling) on each multi-band scaled image1172, 1182 to create display images 1174, 1184 (i.e., up-scaled images).The displays 1170, 1180 then display those display images 1174, 1184 forpeople at the first endpoint 1100.

Similarly, the codec 1250 of the second endpoint 1200 receives thetransmission image of the first endpoint from the network 1050 anddecodes the received image 1262 for a backend system 1260. The backendsystem 1260 divides the received image 1262 into scaled images 1272,1282 for two displays 1270, 1280 and performs an inverse nonlinearscaling equation (i.e., up-scaling) on each scaled image 1272, 1282 tocreate display images 1274, 1284 (i.e., up-scaled images). The displays1270, 1280 then display those display images 1274, 1284 for participantsat the second endpoint 1200.

Referring now to FIGS. 2A-2I, an exemplary method 200 for performing anonlinear down-scaling equation is shown by illustrating images atdifferent points in the method 200. The method 200 may be utilized forinstance, in the teleconferencing system of FIG. 1. Referring initiallyto FIGS. 2A and 2B, in this example, the cameras at one endpoint (e.g.,cameras 1110 and 1120 at endpoint 1100 of FIG. 1) each produce an imagethat is 2304 columns×1536 rows of pixels. Therefore, starting out, rawimages 202, 204 are 2304×1536 pixels. In the illustrative example, thecorresponding displays (e.g., displays 1270 and 1280 at endpoint 1200 ofFIG. 1) can display a 1920×1080 pixel image.

Referring to FIGS. 2C and 2D, the dual-axis warping described inInternational Application No. PCT/US2012/041135 is optionally applied tothe image 202 to produce a first starting image 206 with a first numberof columns and rows (e.g., 1920×1080 pixels in this example). Similarly,the dual-axis warping is optionally applied to the image 204 to producea second starting image 208 with a second number of columns and rows(e.g., also 1920×1080 pixels in this example). In practice, the numberof columns, rows or both can be the same or different between the firstand second starting images. Because of the video camera positioningrelative to the participants, there can be a depth of field disparitybetween participants that are seated close to the video camera relativeto participants seated far away from the video camera. The resultingdifferences in perspective cause the participants at the near end of thetable closest to the video camera to appear much larger than theparticipants at the far end of the table (i.e., farthest away from thevideo camera). The dual-axis warping can serve to scale the image aswell as perform image equalization processing that adjusts theperspective of the captured image data such that each participant,regardless of their seating position at the conference table, appears inthe image to be of similar size. Stated another way, the imageequalization distorts the image in a way intended to “undo” the imageartifacts caused by depth of field effects that make objects appearsmaller than they actually are as the object is positioned further backin the scene captured by the image.

For instance, in FIGS. 2A-2B, the circles may represent the heads ofparticipants. In FIG. 2A, the camera would be to the right of the image.Likewise, in FIG. 2B, the camera would be to the left. Notably, theperspective of the cameras causes the represented participants in FIGS.2A and 2B to be larger the closer that participant is to the camera.However, in FIG. 2C, the relative size of the participants has beenequalized relative to FIG. 2A. Likewise, in FIG. 2D, the relative sizeof the participants has been equalized relative to FIG. 2B. Moreover,the image resolution has been adjusted from 2304×1536 to 1920×1080.

Referring to FIGS. 2E and 2F, in this example, the first starting image206 is divided into three sections (i.e., bands) 210, 212, 214, and eachsection is 1920 x 360 pixels in height. Likewise, for sake of example,the second starting image 208 is divided into three sections (i.e.,bands) 216, 218, 220, and each section is 1920×360 pixels. However, thedivision of a starting image into two or more bands does not have to beeven. Rather, the starting image can be divided up into many bands, eachhaving unique dimensions if so desired.

The middle section 212 is determined to be an area of interest of thefirst starting image 206, so a scaling function that does not reduce thenumber of rows of pixels by too much is chosen for that section.However, the outer sections 210, 214 are to be scaled more to make upfor the relatively low amount of scaling in the middle section 212. Ananalogous set of considerations is applied to the second starting image208, where the middle section is 218 is determined to be an area ofinterest so the middle section 218 will have less scaling than the outersections 216 and 220.

Referring to FIGS. 2G and 2H, for example, for the first starting image206, a scaling function that scales down the image to have three-fourthsthe number of rows of pixels is applied to the middle section 212, and ascaling function that scales down the image to have three-eighths thenumber of rows of pixels is applied to the outer sections 210, 214.Thus, in a first multi-band scaled image 222 the scaled middle section228 has two-hundred-and-seventy (270) rows of pixels, and the outerscaled sections 226, 230 both have one-hundred-and-thirty-five (135)rows of pixels. Therefore, the first multi-band scaled image 222 is1920×540 pixels. Thus, the number of pixels of the multi-band scaledimage 222 is half of the number of pixels of the starting image 206 inthis illustrative example. Likewise, for the second starting image 208,a scaling function is applied that is analogous to that applied to thefirst starting image 206. Thus, in a second multi-band scaled image 224the scaled middle section 234 has two-hundred-and-seventy (270) rows ofpixels, and the outer scaled sections 232, 236 both haveone-hundred-and-thirty-five (135) rows of pixels. Therefore, the secondmulti-band scaled image 224 is also 1920×540 pixels.

However, in the multi-band scaled images, the area of interest (e.g.,the middle section) retains more actual pixels than other areas (e.g.,the outer sections). Therefore, when up-scaling the area of interest,there is, on average, only one row out of four to up-scale. Further, theother areas are not just cropped off; they can also be up-scaled albeitwith less resolution than the area of interest.

Further, if there were only one section in the image and the image werescaled with a continuous, linear equation, then that image would beconsidered linearly scaled. However, according to aspects of the presentdisclosure, an image is divided into more than one section, and at leastone section is scaled differently than other sections; therefore theimage including more than one section scaled with a nonlinear equation(either continuous or non-continuous) would be nonlinearly scaled. Thestarting image may also be divided into more than three sections. Forexample, the number of sections can be the number of rows in thestarting image or the scaled image. As such the height of each sectionwould be one row. In that case, each row can be scaled differently thana previous or subsequent row. Thus, the entire starting image can benonlinearly scaled.

Referring now to FIG. 21, the two scaled images 222, 224 are combined toproduce one transmission image 242 that is 1920×1080 pixels by stackingthe first scaled image 222 on top of the second scaled image 224, orvice versa. The transmission image 242 is sent to a single codec forencoding and for transmission to the other endpoint. Once on the otherendpoint, the images are up-scaled and displayed on separate displays,as described with reference to FIG. 1.

Note that the down-scaling described herein is not a “compression”algorithm performed by the codec. The down-scaling (i.e., reducing theoverall number of rows) occurs before the image reaches the codec.However, the codec may process the combined scaled images further ifdesired (e.g., the codec can compress the transmission image with astandard compression algorithm).

The area of interest may be predefined or may be determined at the timeof scaling. Further, the images may be divided up into any number ofsections for scaling. Moreover, the sections do not necessarily need tobe created horizontally. In some applications, the identified area ofinterest may result in dividing the starting image into verticalsections. Moreover, the area of interest may cross over severalsections. For example in an image with 1080 sections (one for each rowin the starting image), the area of interest may be 540 sections high.

Still further, any number of images may be used as inputs to the process(i.e., more than the two images described in FIGS. 2A and 2B), and thedual-axis warping (e.g., as described with reference to FIGS. 2C and 2D)is not required. However, aspects of the present disclosure prefer thatthe transmission image (i.e., combined scaled images) be the sameresolution as the displays at the other endpoint; however identicalresolution of the transmission image and displays is not required.

Turning now to FIG. 3, an exemplary dual-axis warping concurrent withnonlinear multi-band scaling is shown. A raw camera image starts out at2304×1536 pixels. A processor performs a dual-axis warping (described ingreater detail in International Application No. PCT/US2012/041135 asmentioned above), which would normally result in an image that is1920×1080 pixels (e.g., the bottom left corner of the warped image at a12.5 degree angle from the bottom right pixel of the raw image and thetop left corner of the warped image at a 4.1 degree angle from the topleft corner of the raw image).

For ease of understanding, the exemplary concurrent nonlinear scalingfunctions are as follows: the top third and bottom third are scaled to135 rows and the middle third is scaled to 270 rows. Thus, the resultingscaled image is 540 rows (i.e., half of 1080 rows). Thus, with theexemplary dual-axis warping function and the exemplary scalingfunctions, the resulting scaled image has a bottom left corner at a 19.4degree angle from the bottom right pixel of the raw image and a top leftcorner of the scaled image at a 12.6 degree angle from the top leftcorner of the raw image. As such, the warping and scaling can occurserially (e.g., as described with reference to FIGS. 2A-2I) orconcurrently (e.g., as described with reference to FIG. 3).

Referring now to FIG. 4, an exemplary, simplified method 400 fornonlinear down-scaling is shown. At 402, a first starting image isreceived. The first starting image is an image that will be nonlinearlyscaled according to aspects of the method 400. For example, the firststarting image may be a raw image directly from a camera, a warped imagefrom a dual-axis warping function, an already-scaled image that is to bescaled again, a cropped image, combinations thereof, etc.

At 404, the first starting image is divided into bands (i.e., sections).For instance, as illustrated with reference to FIGS. 2A-2I and FIG. 3,the first starting image is divided into a first starting section, asecond starting section, and a third starting section. The dividing canbe horizontal (such that the number of columns for each section is thesame as the number of columns of the image) or vertical (such that thenumber of rows for each section is the same as the number of rows of theimage). Further, the resulting sections may all be the same size or maybe different sizes. Moreover, the determination of the sizes, shapes, orboth of the sections may be dependent on identifying an area of interest(or areas of interest) within the image. An area of interest may beconstant for every image (i.e., predetermined) or may vary depending onthe image.

At 406, the first starting section is down-scaled with a first scalingequation to create a first scaled section. Similarly, at 408, the secondstarting section is down-scaled with a second scaling equation to createa second scaled section. Likewise, at 410, the third starting section isdown-scaled with a third scaling equation to create a third scaledsection. The scaling functions may be identical, different, or mixed.For example, the second and third scaling equation may be identical,while the first scaling equation is different from the second and thirdscaling functions. Another example has the first and third scalingequations identical, while the second scaling equation is different fromthe other two. For example, the second scaling equation may reduce thenumber of rows to three-fourths the number of rows, and the first andthird scaling equations may reduce the number of rows to three-eighthsof the number of rows.

As mentioned above, the dividing into three sections is just an exampleand used to illustrate the process. However, more sections may be used,limited only by the number of rows in the starting image or the numberof rows in the scaled image.

At 412, the first scaled section, second scaled section, and thirdscaled section are combined to create a first scaled image. Thecombination of the sections may depend on the division of the startingimage. For example, if the starting image were divided horizontally at404, then the scaled sections would be stacked vertically to create thescaled image at 412. Alternatively, if the starting image were dividedvertically at 404, then the scaled sections would be stackedside-by-side to create the scaled image at 412.

In some cases, a second starting image goes through a similar process400 to create a second scaled image. Then the first scaled image andsecond scaled image are combined to create a transmission image that isencoded and sent to another location. The scaled images are up-scaled atthe other location with minimal distortion of the resolution of thefinal image for display.

Thus, with certain scaling functions, two images may be scaled to useonly half the number of pixels and thus half of the information to betransmitted. Therefore, two images may be combined into one image andtransmitted, requiring bandwidth associated with only one image. Thus,two cameras feeding a single encoder at one endpoint can send imagesusing only the bandwidth associated with sending one image normally.

FIGS. 5-6 illustrate a method for video conferencing. Specifically, FIG.5 illustrates preparing images for transmission, and FIG. 6 illustratesreceiving transmitted images and displaying those images.

Turning specifically to FIG. 5, a method 500 for preparing messages fortransmission in video teleconferencing is shown. A first starting imageand a second starting are received at 502 and 504 respectively. Asmentioned above, the starting images may be raw images directly fromcameras or images that have been previously processed (e.g., scaled,warped, cropped, etc., or combinations thereof).

The first starting image is nonlinearly down-scaled to create a firstscaled image at 506. Likewise, the second starting image is nonlinearlydown-scaled to create a second scaled image at 508. As such, thestarting images are down-scaled using a nonlinear down-scaling equation.

Further, the down-scaling may also include a dual-axis warping, asdescribed in International Application No. PCT/US2012/041135. As such,the dual-axis warping may be performed serially (i.e., before or after)with the down-scaling. In other words, the down-scaling/dual-axiswarping process may include two (or more) equations: at least onenonlinear equation (several linear or nonlinear equations may becombined to create one nonlinear continuous equation or nonlinearnon-continuous equation) for down-scaling and at least one equation fordual-axis warping. Alternatively, the dual-axis warping may be performedconcurrently with the down-scaling. In other words, there may be oneequation that implements both dual-axis warping and down-scaling, theremay be more than one equation running concurrently: at least oneequation for down-scaling and at least one equation for dual-axiswarping, or there may be a combination thereof

The scaled images will have fewer rows per image than the startingimages, because the starting images are down-scaled to create the scaledimage. In some instances, the down-scaling equation produces a scaledimage with half the number of rows as the starting image. However, anynumber of resulting rows is acceptable, as long as the number of rows inthe scaled image is less than the number of rows in the starting image.If the down-scaling is performed vertically, then the number of columnsof the scaled image will be less than the number of columns of thestarting images, but the number of rows will not necessarily be less(depending on whether dual-axis warping is also performed).

At 510, the first scaled image and the second scaled image are combinedto create a transmission image. For example, the second scaled image maybe placed on top of the first scaled image (or vice versa) such that theresulting transmission image has a number of rows equal to the number ofrows of the first scaled image plus the number of rows of the secondscaled image. If vertical down-scaling is used, then the scaled imagesmay be placed side-by-side such that the transmission image has a numberof columns as the sum of the columns of the first and second scaledimages.

At 512, the transmission image is compressed by a codec fortransmission. The down-scaling and compression are not the same process.For example, the nonlinear down-scaling is not a standard compressionalgorithm (e.g., H.264, H.265, etc.). Further, the down-scaling may beimplemented independently of the codec used for compression and beforecompression occurs. The compressed transmission image is thentransmitted at 514. In implementations without compression, thetransmission image is transmitted without compression.

In FIG. 6, a method 600 for receiving and displaying images for videoteleconferencing is shown. At 602 the transmission image that wastransmitted at 514 of FIG. 5 is received. If the transmission image wascompressed before transmission, then the compressed transmission imageis decompressed after receipt.

At 604, the received transmission image is separated into a firstdown-scaled image and a second down-scaled image. If the transmissionimage was not compressed or if the transmission image was compressedwith a lossless compression algorithm, then the first down-scaled imageshould be identical to the first scaled image created in 506 of FIG. 5and the second down-scaled image should be identical to the secondscaled image created in 508 of FIG. 5. However, there may be minordifferences between the images stated above if the transmission imagewas compressed using a lossy algorithm.

The first down-scaled image is up-scaled to create a first up-scaledimage at 606, and the second down-scaled image is up-scaled to create asecond up-scaled image at 608. For example, the images may be up-scaledusing an equation that is the inverse of the equation used fordown-scaling. Alternatively, an up-scaling equation independent of thedown-scaling equation can be used if desired. If dual-axis warping wasperformed, then the dual-axis warping remains. As with the down-scaling,the up-scaling is performed independently of the codec.

At 610, the first and second up-scaled images are displayed. Forexample, the first up-scaled image is displayed on a first display, andthe second up-scaled image is displayed on a second display.

Referring to FIG. 7, a block diagram of a data processing system isdepicted in accordance with the present invention. The data processingsystem may implement the front end (e.g., 1140, 1240 of FIG. 1), theback end (e.g., 1160, 1260 of FIG. 1), the codecs (e.g., 1150, 1250 ofFIG. 1), or combinations thereof. The data processing system may also beused to implement any of the methods described more fully herein. Thedata processing system 700 includes one or more processors 702 that areconnected to memory 704 via a system bus 706. The processor(s) 702 maycomprise microprocessors, a symmetric multiprocessor (SMP) system,special purpose or specially programmed devices such as an ApplicationSpecific Integrated Circuit (ASIC), Field Programmable Gate Array(FPGA), etc. A bridge 708 connects the system bus 706 to an I/O Bus 710that links peripheral devices to the processor(s) 702. Peripherals mayinclude storage 712, such as a hard drive, removable media storage 714,e.g., floppy, flash, CD and/or DVD drive, I/O device(s) 716 such as akeyboard, mouse, etc. and a network adapter 718. The memory 704, storage712, removable media insertable into the removable media storage 714 orcombinations thereof, can be used to implement the methods,configurations, interfaces and other aspects set out and describedherein with regard to FIGS. 1-6.

As will be appreciated by one skilled in the art, aspects of the presentdisclosure may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present disclosure may take theform of an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present disclosure may take the form of acomputer program product embodied in one or more computer readablestorage medium(s) having computer readable program code embodiedthereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM), Flash memory,an optical fiber, a portable compact disc read-only memory (CD-ROM), anoptical storage device, a magnetic storage device, or any suitablecombination of the foregoing. In the context of this document, acomputer readable storage medium may be any tangible medium that cancontain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device. A computer readablestorage medium explicitly does not include anything defined as acomputer readable signal medium.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device. As such, a computerreadable storage medium is mutually exclusive from a computer readablesignal medium.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object oriented programming language(e.g., Java, Smalltalk, C++, etc.), conventional procedural programminglanguages (e.g., the “C” programming language or similar programminglanguages), and a hardware description language (e.g., Verilog, VHDL(VHSIC (Very High Speed Integrated Circuit) Hardware DescriptionLanguage), AHDL (Altera Hardware Description Language), etc.). Theprogram code may execute entirely on the user's data processing system,partly on the user's data processing system, as a stand-alone softwarepackage, partly on the user's data processing system and partly on aremote data processing system (e.g., combination of the front end 1140and backend 1260 of FIG. 1) or entirely on the remote data processingsystem or a server. In the latter scenario, the remote data processingsystem may be connected to the user's data processing system through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).

Aspects of the present disclosure are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices (e.g., field programmable gatearrays (FPGA), complex programmable logic devices (CPLD), etc.) tofunction in a particular manner, such that the instructions stored inthe computer readable medium produce an article of manufacture includinginstructions which implement the function/act specified in the flowchartand/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices (FPGAs,CPLDs, etc.) to cause a series of operational steps to be performed onthe computer, other programmable apparatus, or other devices to producea computer implemented process such that the instructions which executeon the computer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a,” “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises,”“comprising,” “includes,” and/or “including” when used in thisspecification, specify the presence of stated features, integers, steps,operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Aspects ofthe invention were chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method for video teleconferencing, the methodcomprising: receiving a first starting image with a first number ofrows; receiving a second starting image with a second number of rows;down-scaling, with a processor, the first starting image to create afirst scaled image with a number of rows fewer than the first number ofrows of the first starting image, wherein the down-scaling is nonlineardownscaling; down-scaling, with the processor, the second starting imageto create a second scaled image with a number of rows fewer than thesecond number of rows of the second starting image, wherein thedown-scaling is nonlinear downscaling; and combining the first scaledimage with the second scaled image to create a transmission image. 2.The method of claim 1 further including: compressing the transmissionimage; and transmitting the compressed transmission image.
 3. The methodof claim 2, wherein: compressing the transmission image includescompressing the transmission image with a codec; down-scaling the firststarting image includes down-scaling the first starting imageindependently of the codec; and down-scaling the second starting imageincludes down-scaling the second starting image independently of thecodec.
 4. The method of claim 1 further including: transmitting thetransmission image; receiving the transmitted transmission image;separating the received transmission image into a first down-scaledimage and a second down-scaled image; up-scaling the first down-scaledimage to create a first up-scaled image; up-scaling the seconddown-scaled image to create a second up-scaled image; displaying thefirst up-scaled image; and displaying the second up-scaled image.
 5. Themethod of claim 4, wherein: displaying the first up-scaled imageincludes displaying the first up-scaled image on a first display; anddisplaying the second up-scaled image includes displaying the secondup-scaled image on a second display.
 6. The method of claim 4 furtherincluding: compressing the transmission image with a first codec,wherein the compressing occurs after the down-scaling; and decompressingthe received transmission image with a second codec, wherein thedecompressing occurs before the up-scaling.
 7. The method of claim 4,wherein: down-scaling the first starting image further includesdown-scaling the first starting image with a nonlinear down-scalingequation; and up-scaling the first down-scaled image further includesup-scaling the first down-scaled image using an inverse of the nonlineardown-scaling equation.
 8. The method of claim 1, wherein: down-scalingthe first starting image further includes performing dual-axis scaling;and down-scaling the second starting image further includes performingdual-axis scaling.
 9. The method of claim 8, wherein: performingdual-axis scaling on the first image occurs concurrently withdown-scaling the first image; and performing dual-axis scaling on thesecond image occurs concurrently with down-scaling the second image. 10.The method of claim 8, wherein: performing dual-axis scaling on thefirst image occurs serially to down-scaling the first image; andperforming dual-axis scaling on the second image occurs serially todown-scaling the second image.
 11. The method of claim 1, wherein:down-scaling the first starting image includes down-scaling the firststarting image to create the first scaled image with a number of rowsequal to half of the number of rows of the first starting image; anddown-scaling the second starting image includes down-scaling the secondstarting image to create the second scaled image with a number of rowsequal to half of the number of rows of the second starting image. 12.The method of claim 1, wherein: combining the first scaled image withthe second scaled image further includes stacking the first scaled imageon top of the second scaled image to create the transmission image. 13.A method for video teleconferencing, the method comprising: receiving afirst starting image with a first number of columns; receiving a secondstarting image with a second number of columns; down-scaling, with aprocessor, the first starting image to create a first scaled image witha number of columns fewer than the number of columns of the firststarting image, wherein the down-scaling is nonlinear downscaling;down-scaling, with the processor, the second starting image to create asecond scaled image with a number of columns fewer than the number ofcolumns of the second starting image, wherein the down-scaling isnonlinear downscaling; and combining the first scaled image with thesecond scaled image to create a transmission image.
 14. The method ofclaim 13, wherein: combining the first scaled image with the secondscaled image further includes stitching the second scaled image besidethe first scaled image to create the transmission image.
 15. A computerreadable storage device with an executable program for videoteleconferencing stored thereon, wherein the program instructs aprocessor to perform: receiving a first starting image with a firstnumber of rows; receiving a second starting image with a second numberof rows; down-scaling the first starting image to create a first scaledimage with a number of rows fewer than the first number of rows of thefirst starting image, wherein the down-scaling is nonlinear downscaling;down-scaling the second starting image to create a second scaled imagewith a number of rows fewer than the second number of rows of the secondstarting image, wherein the down-scaling is nonlinear downscaling; andcombining the first scaled image with the second scaled image to createa transmission image.